


April 2017 


What's Known 


What is the evidence base to 
support reading interventions 
for improving student 
outcomes in grades 1-3? 



Russell Gersten 
Rebecca Newman-Gonchar 
Kelly S. Haymond 
Joseph Dimino 

Instructional Research Group 


ey findings 




This comprehensive review of research literature from 2002 to June 2014 assessed 
the evidence base supporting reading interventions in grades 1-3 to improve 
reading outcomes for students at risk of struggling with typical classroom reading 
instruction. The findings are based on studies of 20 interventions that the review 
team identified and determined met What Works Clearinghouse evidence standards. 
Among the findings: 

All but 1 of the 20 reading interventions demonstrated positive or potentially 
positive effects in at least one area of reading performance. Effects were 
strongest and most consistent in word and pseudoword reading, though several 
interventions also had effects in reading comprehension and passage reading 
fluency. No effects were found in vocabulary. 

All 11 individually administered interventions and 8 of the 9 small-group 
interventions had positive or potentially positive effects. 

All 20 interventions included high levels of ongoing support for teachers, 
paraeducators, volunteers, and other adults who worked with students. 
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Summary 


Response to intervention (RTI) is a comprehensive early detection and prevention strategy 
used to identify and support struggling students before they fall behind. An RTI model 
usually has three tiers or levels of support. Tier 1 is generally defined as classroom instruC' 
tion provided to all students, tier 2 is typically a preventive intervention offered to stu- 
dents who fall behind when given only classroom instruction, and tier 3 is more intensive 
intervention offered to students who failed to respond to the supports in tiers 1 and 2. 

This review provides updated information on the evidence supporting the use of reading 
interventions for students who are at risk of reading difficulty in grades 1-3. The review 
was conducted by Regional Educational Laboratory Southeast in response to discussions 
with members of its Improving Literacy Research Alliance. Alliance members became 
even more interested in the topic after a recently completed national evaluation using 
intensive reading interventions in an RTI model failed to show positive impacts for stu- 
dents who scored at or slightly below the score that would make them eligible for RTI 
services in their school (Balu et al, 2015). 

The review team conducted a comprehensive review of the research literature from 2002 
(the year that the No Child Left Behind Act went into effect and triggered large-scale 
national implementation of reading interventions) to June 2014, when this study started. 
The purpose of the review was to assess the current evidence base on the use of reading 
interventions for improving student outcomes in grades 1-3. The review was limited to 
studies of tier 2 interventions, those designed to provide preventive services to students 
at risk of struggling with typical classroom reading instruction. It did not include studies 
whose subject was intensive (tier 3) intervention — that is, studies geared to students who 
require more than tier 2 support. 

The literature search and review identified 27 efficacy studies 1 that the review team deter- 
mined met What Works Clearinghouse (WWC) evidence standards either with or without 
reservations (What Works Clearinghouse, 2014a). Of the 27 studies, 23 compared the per- 
formance of students who received the intervention with the performance of students who 
did not. (Some interventions were examined in more than one study, and some studies 
examined more than one intervention.) The remaining four studies either explored varia- 
tions in components of one specific intervention or contrasted two interventions, without 
a control condition. Of the 23 studies that compared students who did and those who 
did not receive the intervention, 15 studies examined 13 interventions in grade 1, and 8 
studies examined 7 interventions in grades 2 and 3. Although this report relies heavily on 
WWC protocols, procedures, and standards, and WWC-certified reviewers conducted the 
reviews, this report is not a WWC product. 

Key findings from the 23 efficacy studies of the 20 interventions include: 

• All but 1 of the 20 interventions demonstrated positive or potentially positive 
effects in at least one of the four areas of reading performance: word and pseudo- 
word 2 reading, passage reading fluency, reading comprehension, and vocabulary. 
Effects were strongest and most consistent in word and pseudoword reading, 
though some interventions also had effects in reading comprehension and passage 
reading fluency. No effects were found in vocabulary. 


• All 11 of the individually administered interventions and 8 of 9 of the smalhgroup 
interventions resulted in positive or potentially positive effects. 

• All 20 interventions included high levels of ongoing support for the teachers, para- 
educators, volunteers, and other adults who worked with students. 

Though the reviewed studies showed that 19 of the 20 reading interventions were effec- 
tive, most of the interventions included a component that is atypical of current school 
practice: ongoing support for the interventionist (the teacher, paraeducator, or member of 
the research staff responsible for delivering the intervention). In addition, the majority of 
interventions involved individual (one-on-one) interventions, as opposed to typical school 
implementations, which involve small groups of three to five students. When consider- 
ing how to use these findings, it is important to consider that these studies do not reflect 
typical school practice, where weekly or biweekly monitoring of fidelity of implementation 
and onsite coaching are rarely available. 
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Why this review? 


Over the past 15 years, U.S. schools have engaged in efforts to prevent reading failure 
through early intervention in grades 1-3, a system known as response to intervention 
(RTI). Using brief screening measures, educators identify students in grades 1-3 who are 
likely to experience difficulty learning to read with typical classroom instruction and 
provide them with 20-40 minute interventions, generally in small groups. By intervening 
early and using evidence-based practices whenever possible, educators hope to reduce the 
reading failure rate. Reading instruction provided to all students in a class is generally 
defined as tier 1. Preventive intervention is often called tier 2; students who fail to respond 
to that receive more intensive intervention, defined as tier 3. 


Although several states initiated early reading interventions more than 15 years ago, large- 
scale national implementation began only after Congress passed the No Child Left Behind 
Act (2002) and was further encouraged after Congress passed the Individuals with Dis- 
abilities Education Act (2004). Both laws called on schools to use research-based practices. 
Therefore, many stakeholders were deeply interested in the nature of the evidence base on 
effective reading interventions for students in grades 1-3. 

This review provides updated information on the extent of evidence available to support 
the use of reading interventions for students who are at risk of reading difficulty in grades 
1-3, especially evidence of whether effects could be seen beyond grade 1 and in more 
complex areas of reading such as reading comprehension. Regional Educational Laboratory 
Southeast conducted this review in response to discussions with members of its Improving 
Literacy Research Alliance. Alliance members were even more interested in the topic after 
a recently completed national evaluation using intensive reading interventions in an RTI 
model failed to show positive impacts for students who scored at or slightly below the score 
that would make them eligible for RTI services in their school (Balu et al., 2015). 


This 

comprehensive 
review assesses 
the evidence 
base for reading 
interventions for 
improving student 
reading outcomes 
in grades 1-3 


This comprehensive review covers research literature from 2002 to June 2014 to assess 
the evidence base for reading interventions for improving student reading outcomes in 
grades 1-3. These grades were chosen because, unlike in studies of interventions for kin- 
dergartners, the outcome measures include measures of actual reading performance rather 
than measures of reading-related skills such as phonological awareness, letter naming, or 
listening comprehension. Since the goal of this research synthesis was to report on impacts 
on students’ reading performance, effects on phonological awareness were included only if 
they were accompanied by growth in actual reading outcomes. The goal of this report is to 
provide school and district administrators, school psychologists, school counselors, special 
education teachers, reading specialists, and interventionists (the teachers, paraeducators, 
or members of the research staff responsible for delivering the intervention) with a 
summary of the evidence that supports the use of reading interventions in grades 1-3. 
The review was limited to studies of interventions, those designed to provide preventive 
services to students at risk for reading difficulties (tier 2 interventions). It did not include 
studies involving more intensive interventions geared to students who require more than 
tier 2 support (tier 3 interventions). 
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What the review examined 


This review used a systematic process modeled after the What Works Clearinghouse 
(WWC) study review process, Version 3.0 (What Works Clearinghouse, 2014a). Although 
the review relied heavily on WWC protocols, procedures, and standards and was conduct- 
ed by WWC-certified reviewers, this report is not a WWC product. 

The review addressed one primary research question: 

• What rigorous evidence exists that reading interventions for grades 1-3 demon- 
strate a positive effect on students’ reading performance in the four areas of 
reading performance (word and pseudoword 3 reading, passage reading fluency, 
reading comprehension, and vocabulary)? 4 

To better understand the nature of the interventions studied, the review also addressed 
three descriptive research questions: 

• On what areas of reading or reading-related skills did the interventions focus? 

• What proportion of the interventions used small groups, and how many interven- 
tions were conducted individually? Were effects similar? 

• What level of training did the interventionists receive for implementing the 
intervention? 

The review team then analyzed the studies that it determined met WWC evidence stan- 
dards in three ways: 

1. By calculating the average effect size for each of the four areas of reading performance 
according to WWC procedures to determine the impact of each intervention. 

2. By calculating weighted mean effect sizes to gauge the impact of reading interventions 
across all the interventions for students in grade 1 and in grades 2 and 3. 

3. By examining the descriptions of the interventions and cataloging the intervention 
characteristics to look for patterns that might explain the impacts. 

If an effect is statistically significant, it is highly unlikely that the result was due to chance. 
Statistical significance thus indicates whether an intervention is effective but not how 
effective it is. For that reason, researchers increasingly use effect size to gauge how effec- 
tive the intervention is. An intervention with an effect size of 0.40 is roughly twice as 
effective as one with an effect size of 0.20. To show how effective each intervention is, an 
average effect size can be calculated by averaging the effect on each outcome in each area 
of reading for each intervention. This review indicates which interventions were found 
to demonstrate a statistically significant average effect in any of the four areas of reading, 
which interventions demonstrated a potentially positive 5 average effect with an effect size 
greater than 0.25 that was not statistically significant, and which interventions failed to 
produce such an effect (see box 1 for definitions of the ratings). 

The review also presents weighted mean effect sizes, which combine all the average effect 
sizes for interventions that measured an area of reading, to gauge how effective the full set 
of grade 1 or grade 2 and 3 interventions were in improving student performance in word 
and pseudoword reading, passage reading fluency, reading comprehension, and vocabulary. 


Although this 
review used a 
systematic process 
modeled after 
the What Works 
Clearinghouse 
(WWC) study 
review process, 
this report is not 
a WWC product 
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Box 1. Intervention effects terminology used in this review 


Positive effect. An intervention resulted in the treatment group performing better than the 
comparison group by a statistically significant margin in at least one area of reading in at least 
one study. 

Potentially positive effect. An intervention resulted in the treatment group performing better 
than the comparison group in at least one study, with at least one area of reading having an 
effect size greater than 0.25 that was not statistically significant. 

Negative effect. An intervention resulted in the comparison group performing better than the 
treatment group by a statistically significant margin in at least one area of reading in at least 
one study. 

Potentially negative effect. An intervention resulted in the comparison group performing better 
than the treatment group in at least one study, with at least one area of reading having an 
effect size less than -0.25 that was not statistically significant. 

Inconclusive result. An intervention did not result in any statistically significant effect size or 
an effect size greater than 0.25 or less than -0.25 between the two groups in any area of 
reading in any study. 

Source: Adapted from What Works Clearinghouse (2014a). 


The calculation weights each average based on the number of participants involved in the 
study; therefore, a study with 20 participants would not count as much as one with 120 
participants. 

The initial literature search identified 1,813 studies, which were then screened to deter- 
mine whether they were relevant to the review. The screening identified 43 studies that 
examined tier 2 interventions to improve reading outcomes for grade 1-3 students at risk 
of reading difficulty that were published between 2002 and June 2014, were conducted in 
the United States, and used an experimental or quasi-experimental design (see appendix 
A for details on the search, screening, and review process and appendix B for a list of the 
43 studies that met eligibility screening). Experimental and quasi-experimental studies that 
are done under tightly controlled conditions with careful monitoring of implementation 
and a good deal of support for interventionists are called efficacy studies. Those that are 
conducted in situations more aligned with typical school practice are called effectiveness 
trials. 


The 27 studies that 
the review team 
determined met 
WWC evidence 
standards 
examined 
20 reading 
interventions. 

This report 
focuses on the 
characteristics and 
findings of those 
20 interventions 


Of the 43 studies that met eligibility screening, the review team determined that 27 effi- 
cacy studies 6 met WWC evidence standards (Version 3.0) either with or without reserva- 
tions (figure 1; see appendix C for summary tables describing the studies that met WWC 
evidence standards) and that 16 studies did not (see table A1 in appendix A). Of the 27 
studies that the review team determined met WWC evidence standards, 23 examined the 
impact of an intervention by comparing the performance of students who received the 
intervention with the performance of students who did not (a control condition). The 23 
studies examined 20 reading interventions (some interventions were examined in more 
than one study, and some studies examined more than one intervention). This report 
focuses on the characteristics and findings of the 20 interventions that had studies that the 
review team determined met WWC standards. 
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Figure 1. The review team determined that 27 studies met What Works 
Clearinghouse evidence standards and that 22 studies demonstrated positive or 
potentially positive effects of 19 reading interventions on reading performance 



Met What Works Clearinghouse evidence 
standards with or without reservations, 
as determined by the review team 


Showed positive or potentially positive effects on 
word and pseudoword reading, passage reading 
fluency, reading comprehension, or a combination 


Note: Vocabulary was included in the search, but no studies found positive or potentially positive effects. 
Source: Authors' analysis of primary data collected for the review; see appendix A. 


The remaining 4 studies (that is, 4 of the 27 studies that met WWC evidence standards) 
either explored variations in components of one specific intervention (see table C3 in 
appendix C) or contrasted two interventions without a control condition (see table C4 
in appendix C). Findings from these four studies are used for supplemental purposes. This 
report, however, focuses on the impact of the 20 interventions from the 23 studies. 

What the review found 


Of the 20 
interventions, 19 
produced positive 
or potentially 
positive effects in 
at least one area of 
reading; none had 
negative effects 


Of the 20 interventions represented in the 23 studies that the review team determined met 
WWC standards, 19 produced positive or potentially positive effects in at least one area of 
reading; none had negative effects (see figure 1). 

Twelve of the thirteen grade 1 interventions and all seven grade 2 and 3 interventions had positive 
or potentially positive effects 

Thirteen were grade 1 interventions. Of these, eight produced positive effects in at least 
one of the four areas of reading performance, and four produced effects considered poten- 
tially positive in at least one of the four areas of reading performance. Four of the 12 also 
had inconclusive findings in another area of reading. One of the 13 interventions had only 
inconclusive findings (and no positive or potentially positive effects; table 1). 

Five of the seven grade 2 and 3 interventions produced positive effects, and two produced 
only potentially positive effects. None resulted in only inconclusive findings (table 2). 
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Table 1. Effects of grade 1 reading interventions by area of reading performance 


Intervention 

Word and 
pseudoword 
reading 

Passage 

reading 

fluency 

Reading 

comprehension Vocabulary 2 

Positive effects 

Early Literacy Tutoring Program (Allor & 
McCathren, 2004) 

• 

■ 

• 

Intensive Tutorial Intervention (Scanlon, 
Vellutino, Small, Fanuele, & Sweeney, 2005) 

• 


• 

Interactive Strategies (Vellutino & Scanlon, 

2002) • 

Phonics-based Early Reading Intervention 
(Vadasy & Sanders, 2011) 

• 

• 

O 

Reading Recovery (Schwartz, 2005) 

• 


■ 

Researcher-modified Sound Partners (Jenkins, 
Peyton, Sanders, & Vadasy, 2004) 

• 


• 

Responsive Reading Instruction (RRI; Denton 
et at, 2010) 

• 


• 

University of Florida Literacy Initiative (Lane, 

Pullen, Hudson, & Konold, 2009) • 

Potentially positive effects 

BRIO Supplemental Intervention (Wang & 
Algozzine, 2008) 

O 


■ 

Combination of Fundations, Responsive 

Reading Instruction, and Read Naturally (Case 
et al., 2010, 2014) 

O 

■ 


Reading intervention developed by Fuchs, 
Compton, Fuchs, Bryant, & Davis (2008) 

o 



Volunteer Tutoring (Pullen, Lane, & Monaghan, 
2004) 

o 



Inconclusive results 

Reading intervention developed by Wanzek & 
Vaughn (2008) 

■ 

■ 



• Produced positive effects. O Produced potentially positive effects. ■ Produced inconclusive results, 
a. No grade 1 interventions had outcome measures in vocabulary. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 


Among the four areas of reading, effects were strongest and most consistent in word and 
pseudoword reading 

Weighted mean effects were strongest in word and pseudoword reading (0.452 for grade 1 
interventions and 0.456 for grade 2 and 3 interventions; figure 2; see also tables D1 and D2 
in appendix D). Twelve of the thirteen grade 1 interventions and all of the seven grade 2 
and 3 interventions had studies that found positive or potentially positive effects in word 
and pseudoword reading (see tables Cl and C2 in appendix C). Word and pseudoword 
reading also had the most consistent results; only one grade 1 intervention had a study 
with inconclusive findings. 

More than half of the grade 1 interventions and all but one of the grade 2 and 3 inter- 
ventions were assessed using a reading comprehension measure. For grade 1 interventions, 
effects were lower in reading comprehension (0.386) than in word and pseudoword reading 
(0.452) but higher than in passage reading fluency (0.226). For grade 2 and 3 interventions, 
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Table 2. Effects of grade 2 and 3 reading interventions by area of reading 
performance 

Intervention 

Word and 
pseudoword 
reading 

Passage 

reading 

fluency 

Reading 

comprehension Vocabulary 

Positive effects 

Explicit, sequential instruction in word reading 
and comprehension (Denton, Fletcher, Taylor, 
Barth, & Vaughn, 2014) 

• 

• 

O 

Guided Reading (Denton et al., 2014) 

• 

■ 

■ 

Reading intervention developed by Blachman 
et al. (2004) 

• 

• 

• 

Reading Mastery/Corrective Reading (Gunn, 
Smolkowski, Biglan, Black, & Blair, 2005) 

• 

■ 

• ■ 

Reading to an adult listener (O'Connor, 
Swanson, & Geraghty, 2010) 

O 

• 

o ■ 

Potentially positive effects 

Before/After School Reading Club (Berninger, 
Abbott, Vermeulen, & Fulton, 2006) 

O 



Paraeducator-supplemented instruction 
(Vadasy, Sanders, & Peyton, 2006; Vadasy, 
Sanders, & Tudor, 2007) 

o 

O 

o 

• Produced positive effects. O Produced potentially positive effects. ■ Produced inconclusive results. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 



Figure 2. For both grade 1 reading interventions and grade 2 and 3 reading 
interventions, weighted mean effect sizes were highest for word and pseudoword 
reading 


Weighted mean effect size 



a. No grade 1 interventions had outcome measures in vocabulary. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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the weighted mean effects in reading comprehension (0.327) were lower than those for 
word and pseudoword reading (0.456) and passage reading fluency (0.374). Of the seven 
grade 1 interventions with a measure of reading comprehension, four had positive effects, 
and one had a potentially positive effect. For the six grade 2 and 3 interventions with a 
reading comprehension measure, two had positive effects, and three had potentially pos- 
itive effects. Thus, some evidence suggests that these interventions often have positive 
effects in reading comprehension. 


The effects were lower in passage reading fluency (0.226) than in word and pseudoword 
reading and reading comprehension for grade 1 interventions. For grade 2 and 3 interven- 
tions, the effects in passage fluency (0.374) were lower than those for word and psuedoword 
reading, but higher than those for reading comprehension. The weighted mean effect size 
in passage reading fluency was significant for grade 1 interventions delivered individually 
but not for grade 1 interventions delivered in small groups (see table D1 in appendix D). 
Four of the grade 1 interventions included a measure of passage reading fluency: one had 
positive effects, and three had inconclusive findings. Six of the grade 2 and 3 interventions 
included a measure of passage reading fluency: three had positive effects, one had poten- 
tially positive effects, and two had inconclusive findings. Thus, evidence of effectiveness in 
passage reading fluency is limited. 

Although most interventions included a vocabulary component, vocabulary was not 
assessed at all in grade 1 and was assessed in only two studies of grade 2 and 3 interven- 
tions. The grade 2 and 3 interventions had a mean weighted effect size that was not statis- 
tically significant for vocabulary. 

Both individually administered and small-group interventions had positive or potentially positive 
outcomes 


All of the 
interventions that 
were administered 
individually 
had positive or 
potentially positive 
effects for both 
grade 1 and grades 
2 and 3. All but 
one of the grade 1 
interventions that 
were administered 
in small groups 
had positive 
or potentially 
positive effects 


Eight of the thirteen grade 1 interventions and three of the seven grade 2 and 3 inter- 
ventions were administered individually, meaning that students met one on one with an 
interventionist (figure 3). All 11 of the interventions that were administered individually 
had positive or potentially positive effects for both grade 1 and grades 2 and 3. 

Five of the thirteen grade 1 interventions and four of the seven grade 2 and 3 interven- 
tions were administered in small groups (figure 4). All but one of the grade 1 interventions 
that were administered in small groups had positive or potentially positive effects. 

Two studies examined whether grouping structure (that is, individually administered 
or small-group structures) had an effect on the findings (see table C3 in appendix C). 
Schwartz et al. (2012) found that grade 1 students who received the Reading Recovery 
intervention individually (as it is intended to be delivered) performed better than students 
who received the intervention in small groups. Vaughn et al. (2003) found no difference 
between the performance of grade 2 students who received a tier 2 reading intervention 
individually and grade 2 students who received the same intervention in small groups. 

All grade 1 and grade 2 and 3 interventions covered multiple areas of reading 

All but one intervention included approximately 10-15 minutes devoted to systematic work 
in decoding (occasionally including sight word practice), and all devoted approximately 
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Figure 3. All individually administered reading interventions had positive or 
potentially positive outcomes for both grade 1 and grades 2 and 3 


Grade 1 

individually administered interventions 



Grades 2 and 3 

individually administered interventions 


JH 

entially 
isitive 
) = 1 

Positi’ 

n = 

k 



w 


Source: Authors' analysis of primary data collected for the review; see appendix A. 


Figure 4. Small-group reading interventions had positive or potentially positive 
effects in all but one case for grade 1 and in all cases for grades 2 and 3 


Grade 1 

small group interventions 



Grades 2 and 3 
small group interventions 


A 




j H 

sntially 
sitive 
= 1 

Positi' 
n = C 

»/e 

3 





Source: Authors' analysis of primary data collected for the review; see appendix A. 


10-15 minutes to passage or sentence reading, sometimes including occasional questions 
to see whether students understood what they were reading. This pattern was fairly similar 
across grade levels and group sizes (tables 3 and 4). 

A larger proportion of time was spent on vocabulary development, comprehension instruc- 
tion, and encoding (spelling) in the grade 2 and 3 interventions than in the grade 1 inter- 
ventions. Slightly fewer than half of the interventions included a writing component. 

None of the variations in lesson content contributed to a meaningful classification system 
for intervention types. In many respects, the interventions appeared to be similar to each 
other, although session length, group size, and amount of training varied, with Reading 
Recovery requiring far more training than the others. A mix of teachers, paraprofession- 
als, and research staff administered the interventions. The amount of training varied 


All but one 
intervention 
included 
approximately 
10-15 minutes 
devoted to 
systematic work 
in decoding, 
and all devoted 
approximately 
10-15 minutes 
to passage or 
sentence reading. 
This pattern was 
similar across 
grade levels and 
group sizes 
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Table 3. Characteristics of grade 1 reading interventions 


Intervention 

Grouping 

Time 




Academic area of instruction 




Implementation 


Intensity and duration 
(intended implementation) 

Total 

instructional time 
(approximate 
hours) 

Passage 
Decoding reading 

Vocab- 

ulary 

Reading 

compre- 

hension 

Phono- 

logical 

aware- 

ness 

Writing 

Encoding Scripted 

Interven- 

tionist 

Hours of 
training 

Ongoing 

support 

BRIC Supplemental Intervention 

Small 

10-15 minute sessions, 5 

23 












(Wang & Algozzine, 2008) 

group 

days per week, for 22 weeks 


✓ 

✓ 



✓ 



✓ 

P 

nr 

✓ 

Combined Fundations, Responsive 

Small 

40 minute sessions, 3 days 

15-16 












Reading Instruction, and Read 

group 

per week, for 11-12 weeks 













Naturally (Case et al., 2010, 2014) 




✓ 

✓ 

✓ 

✓ 

✓ 


✓ 

✓ 

R 

20 

✓ 

Early Literacy Tutoring Program 

Individual 

15-20 minute sessions, 3-4 

10-14 












(Allor & McCathren, 2004) 


days per week, for 17 weeks 


✓ 

✓ 


✓ 

✓ 

✓ 


✓ 

P 

3 

✓ 

Intensive Tutorial Intervention 

Individual 

30 minute sessions, 5 days 

75 












(Scanlon et al., 2005) 


per week, for 30 weeks 


✓ 

✓ 


✓ 

✓ 

✓ 


nr 

T 

35 a 

✓ 

Interactive Strategies (Vellutino & 

Individual 

30 minute sessions, 5 days 

39-78 












Scanlon, 2002) 


per week, for 16-32 weeks 


✓ 

✓ 


✓ 

✓ 

✓ 



T 

30 

✓ 

Phonics-based Early Reading 

Individual 

30 minute sessions, 4 days 

33 












Intervention (Vadasy & Sanders, 


per week, for 20 weeks 













2011) 




✓ 

✓ 





✓ 

✓ 

P 

2 

✓ 

Reading intervention developed by 

Small 

45 minute sessions, 4 days 

27 












Fuchs et al. (2008) 

group 

per week, for 9 weeks 


✓ 

✓ 





✓ 


R 

nr 


Reading intervention developed by 

Small 

30 minute sessions, 5 days 

25 












Wanzek & Vaughn (2008) b 

group 

per week, for 10 weeks 


✓ 

✓ 


✓ 



✓ 

✓ 

R, P 

15 

✓ 

Reading Recovery (Schwartz, 

Individual 

30 minute sessions, 5 days 

40 












2005) 


per week, for 12-20 weeks 


✓ 

✓ 

✓ 

✓ 


✓ 



T 

nr c 

✓ 

Researcher-modified Sound 

Individual 

30 minute sessions, 4 days 

46 












Partners (Jenkins et al., 2004) 


per week, for 25 weeks 


✓ 

✓ 





✓ 

✓ 

P 

3 

✓ 

Responsive Reading Instruction 

Small 

40 minute sessions, 5 days 

62 












(Denton et al., 2010 

group 

per week, for 25 weeks 


✓ 

✓ 


✓ 

✓ 

✓ 

✓ 


T 

18 

✓ 

University of Florida Literacy 

Individual 

32-38 min sessions, 3-4 

23 












Initiative (Lane et al., 2009) 


days per week, for 10 weeks 


✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 


R 

12 

✓ 

Volunteer Tutoring (Pullen et al., 

Individual 

15 minute sessions, 3 days 

9.5 












2004) 


per week, for 12 weeks 


✓ 

✓ 

✓ 

✓ 





P 

4 

✓ 


nr is not reported. T is certified teacher. R is researcher. P is paraprofessional. 

a. Five-day workshop; estimated seven hours per day. 

b. Intervention resulted in inconclusive findings. 

c. Certified Reading Recovery interventionists receive at least a year of graduate-level training and ongoing professional development. 
Source: Authors' analysis of primary data collected for the review; see appendix A. 




Table 4. Characteristics of grade 2 and 3 reading interventions 




Time 




Academic area of instruction 




Implementation 


Intervention 

Grouping 

Intensity and duration 
(intended implementation) 

Total 

instructional time 
(approximate 
hours) 

Passage 
Decoding reading 

Vocab- 

ulary 

Reading 

compre- 

hension 

Phono- 

logical 

aware- 

ness 

Writing 

Encoding Scripted 

Interven- 

tionist 

Hours of 
training 

Ongoing 

support 

Before/After School Reading Club 
(Berninger et al . , 2006) 

Small 

group 

60 minute sessions, 2 days 
per week, for 22 weeks 

44 

✓ 

✓ 




✓ 

✓ 


T, R 

nr 

nr 

Explicit, sequential instruction in 
word reading and comprehension 
(Denton et al., 2014) 

Small 

group 

45 minute sessions, 4 days 
per week, for 23-25 weeks 

59 

✓ 

✓ 

✓ 

✓ 



✓ 

✓ 

P 

27 

✓ 

Guided Reading (Denton et al., 
2014) 

Small 

group 

45 minute sessions, 4 days 
per week, for 23-25 weeks 

59 

✓ 

✓ 

✓ 

✓ 


✓ 

✓ 


P 

27 

✓ 

Paraeducator-supplemented 
instruction (Vadasy et al., 2006, 
2007) 

Individual 

30 minute sessions, 4 days 
per week, for 15-20 weeks 

CD 

CM 

00 

✓ 

✓ 


✓ 



✓ 

✓ 

P 

3 

✓ 

Reading intervention developed by 
Blachman et al. (2004) 

Individual 

50 minute sessions, 5 days 
per week, for 28 weeks b 

105 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 

✓ 


T 

45 

✓ 

Reading Mastery/Corrective 
Reading (Gunn et al., 2005) c 

Small 

group 

30 minute sessions, 3-5 
days per week, for 2 years 

nr 

✓ 

✓ 



✓ 


✓ 

✓ 

P 

10 

✓ 

Reading to an adult listener 
(O’Connor et al., 2010) 

Individual 

15 minute sessions, 3 days 
per week, for 20 weeks 

15 


✓ 

✓ 






P 

2 



nr is not reported. T is certified teacher. R is researcher. P is paraprofessional. 

a. Average hours across three studies with this intervention. 

b. Daily 30 minute sessions for six to seven months during the first year, nine months during the second year, three 30 minute sessions per week for five weeks during the summer between 
years 1 and 2. 

c. Includes parent education for some parents and social behavior training for some students; also K-4. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 




dramatically, from 4 hours to 45 hours, and all but two of the interventions included 
ongoing support for the interventionists. 


Two studies explored variations in content covered in two interventions. Lane et al. (2009) 
examined variations of the grade 1 intervention developed as part of the University of 
Florida Literacy Initiative by deleting one of three components. That study found that 
when the decoding component was excluded, the intervention was no longer effective. 
However, deleting either the writing or the text structure (a form of comprehension 
instruction) component did not reduce the effect. Scanlon et al. (2005) varied the amount 
of decoding instruction per session in the Intensive Tutorial Intervention and found that 
tripling the amount of decoding time per lesson (from 5 minutes to 15) did not increase 
the effect significantly. 

All interventions included implementation support 

In all cases, the interventionist received some training prior to implementing the interven- 
tion, as well as some training over the course of the implementation or feedback or coach- 
ing after being observed implementing the intervention. Most of the interventions included 
scripts or detailed outlines specifying what the interventionist must say and do during each 
lesson. The type of person who served as the interventionist varied from study to study and 
included paraeducators/adult volunteers, certified teachers, and graduate students/researchers. 
All interventions were administered with a large amount of support for interventionists. 

Interventions varied in length 


All interventionists 
received some 
training prior to 
implementing 
the intervention, 
as well as over 
the course of the 
implementation; all 
interventions were 
administered with 
a large amount 
of support for 
interventionists 


Most of the grade 1 interventions and grade 2 and 3 interventions with positive or poten- 
tially positive effects were implemented for 30-45 minutes a day, four to five days a week 
for 12-25 weeks. One was implemented for as much as 50 minutes a day, five days week for 
28 weeks (Blachman et al, 2004), and one was implemented for as few as 15 minutes a day, 
three days a week for 12 weeks (Pullen et al, 2004). Only one study (Denton et al, 2014) 
examined variations in an intervention’s duration and intensity; it found inconclusive results. 

Implications of the findings 


The majority of interventions with studies that the review team determined met WWC 
standards had positive or potentially positive effects in at least one area of reading per- 
formance. Early interventions (particularly in grade 1) can lead to positive effects in word 
and pseudoword reading and in reading comprehension. Too few studies assessed passage 
reading fluency or vocabulary to draw any inferences. 

However, the conditions in most of the studies do not reflect common RTI practice in 
schools (Balu et al., 2015), which is almost always small-group instruction rather than the 
individual instruction provided in the majority of grade 1 interventions examined in the 
identified studies. These results do not suggest that individual instruction is superior to 
small-group instruction, but only that researchers have done far more research on this 
approach, especially in grade 1. Also, virtually all studies included in this review indicated 
that a lot of ongoing support was provided for implementation. It is unclear how generaliz- 
able these findings are when the typical amount of ongoing support for interventionists is 
far more limited in practice. 
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Early intervention in reading that includes decoding words and passage reading is an evidence- 
based practice 


Twenty-two of the studies that were reviewed provide evidence for the use of tier 2 inter- 
ventions in reading for students who are falling behind in grades 1-3. There is some evi- 
dence that the most promising interventions devote time to decoding words and to passage 
reading. Two studies examined the impact of including different amounts of decoding 
practice. Both suggest that, although time for decoding is likely very useful for students 
who are still learning to associate sounds with letters, doubling or tripling the amount of 
time may not necessarily be helpful (Lane et al., 2009; Scanlon et al, 2005). 


Interventions will most likely improve word and pseudoword reading in grades 1-3 

In general, effects were most prevalent and strongest in word and pseudoword reading for 
both grade 1 interventions and grade 2 and 3 interventions. Results were also positive but 
less prevalent for reading comprehension at both grade levels. Results for passage reading 
fluency were positive only for grade 2 and 3 interventions. Very few studies looked at 
vocabulary, and none found effects in that area. 

Both small-group and individually administered interventions demonstrated positive effects 

Slightly more than half of the interventions (11 of the 20) examined in studies that met 
WWC evidence standards were administered individually, but results of both small-group 
and individually administered interventions were found to have positive or potentially pos- 
itive effects in at least one area of reading — with only one exception. 


In general, 
effects were most 
prevalent and 
strongest in word 
and pseudoword 
reading for 
both grade 1 
interventions and 
grade 2 and 3 
interventions 


Research has not examined less intensive support for interventionists 

The majority of interventions studied provided a large amount of ongoing support for 
interventionists. Because studies of interventions with less support were not found during 
the search process, no conclusions can be drawn regarding levels of support. 

Limitations of the review 


The findings are based on a large body of research, but some types of research literature 
that may be relevant to the field were excluded from the review, including gray litera- 
ture (dissertations and reports from school districts, state departments, or commercial 
publishers). 

This search excluded interventions that were shorter than eight hours and those that 
targeted a specific student population (such as English learner students or students with 
disabilities). These interventions may be relevant to the concerns of other districts. For 
information on interventions for English learner students, see Baker et al. (2014). For infor- 
mation on interventions for students with disabilities, see the WWC website (http://ies. 
ed.gov/ncee/wwc/FWW/Results?filters=,Children'Youth'with'Disabilities). 

The literature search concluded in June 2014. More recent research may provide additional 
information relevant to RTI practice in reading, as well as WWC intervention reports and 
single-case study reviews. Relevant WWC resources on this topic, which include a Single 
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Study Review of a Reading Recovery study, are available on the WWC website (http://ies. 
ed.gov/ncee/wwc/Study/32027). 

Many other reading interventions exist beyond those presented here. Only interventions 
with studies that the review team determined met WWC evidence standards, Version 3.0, 
with or without reservations are included here. This report highlights these interventions 
because the research evidence is of the highest quality in determining the intervention’s 
effect on student achievement, which is just one, albeit important, piece of evidence to 
consider when deciding whether to adopt an intervention. 

The effect size calculations are limited to the small number of studies that met WWC evi- 
dence standards. A larger number of studies would increase the accuracy and decrease the 
variability of the results. As research continues to make progress in evaluating RTI and 
reading, summaries like the current review should be updated. 

Studies that the review team rated as meeting WWC evidence standards can provide sci- 
entific evidence to support the intervention only as implemented in the study, with the 
sample and setting specified in the study. Although results suggest that many reading 
interventions are effective, most of the interventions included ongoing support for inter- 
ventionists, a situation not typical of current school practice. Additionally, the majority 
involved individual (one-on-one) interventions, as opposed to typical school implemen- 
tations, which usually involve small groups of three to five. When considering how to use 
these findings, it is important to remember that these studies do not reflect typical school 
practice, but rather tightly controlled implementation with a high degree of support for the 
intervention ists . 
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Appendix A. The search, screening, and review process 


This appendix describes the literature search, screening, and review processes used in this 
report. 

Literature search 

To locate potentially eligible studies, the review team conducted a literature search using 
the following steps: 

• A keyword search (see box A1 for keywords) was conducted using the following 
databases: 

o Academic Search Premier. 

° Campbell Collaboration, 
o Educator’s Reference Complete, 
o ERIC. 

° PsycINFO. 

o Social Sciences Citation Index, 
o WorldCat. 

• The review team also solicited key researchers (for example, Stephanie AbOtaiba, 
Scott Baker, Don Compton, Michael Coyne, Jack Fletcher, Barbara Foorman and 
Florida Center for Reading Research staff, Doug and Fynn Fuchs, and Sharon 
Vaughn) for recommendations of studies likely to meet eligibility criteria. 


Box Al. Keywords used 

Reading words 

Reading 

Literacy 

Fluency 

Decoding 

Vocabulary 

Comprehension 

Reading ability 

Reading proficiency 

Reading achievement 


Response to intervention and intervention 

Reading intervention 


in database searches 


RTI 

Response to intervention 
Response to instruction 
Tier 2 intervention 
Tier 3 intervention 
Tutoring 

Small-group instruction 
One-on-one instruction 
Intensive intervention 


At-risk students 

At-risk 

Continued risk 
Non-responders 
Responders 
Reading difficulties 
Reading disabilities 
Struggling readers 

Grade 

Grade 1 

First grade 

Grade 2 

Second grade 

Grade 3 

Third grade 

Primary grades 

Elementary grades 

Early elementary grades 

Early literacy 

Location 

United States 
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• The references of meta-analyses, literature reviews focused on reading inter- 
ventions for grades 1-3, and What Works Clearinghouse (WWC) intervention 
reports in beginning reading were reviewed and cross-referenced with the previous 
results of the literature search to identify any additional studies that might not 
have been captured. 

Screening process and study eligibility criteria 

The study abstracts and full reports of the 1,813 studies identified through the literature 
search were screened for eligibility. Studies that were eligible for review met the following 
relevancy criteria: 

Study design. Studies used a randomized controlled trial or quasi-experimental design. 

Topic. Studies examined a small-group or individual (one-on-one) intervention program 
aimed at helping students who are considered at risk of reading difficulties. The interven- 
tion was not required to be a part of a fully developed response to intervention or multi- 
tiered system of support. 

Publication status and date. Studies were published in a peer-reviewed journal between 
January 1, 2002, and June 30, 2014. 

Sample. Studies included a student sample that met the following criteria: 

• Grade levels. The students at risk of reading difficulties in the study sample were 
in grades 1-3. The review did not include studies that contained students in other 
grades (such as K or upper elementary) unless the study findings disaggregated the 
results of students in eligible grades or students in eligible grades represented more 
than 50 percent of the aggregated mixed-age sample. 

• General education students. At least 60 percent of the students in each study were 
general education students and not classified as an English learner student or a 
student with disabilities. 

• Location of the intervention. The samples all resided in the United States or its 
territories. 

• Achievement level. Students were considered at risk of reading difficulties if they 
met one of the following criteria: 

° A score on either a valid screener (such as AIMSweb or Dynamic Indicators 
of Basic Early Literacy Skills Next) or screening battery (such as the Texas 
Primary Reading Inventory; Foorman, Fletcher, & Francis, 2014) indicated 
that the student was likely to be at risk at the end of the school year. 

° A score on a norm-referenced standardized test (such as Woodcock Reading 
Mastery) indicated that a student performed below the 40th percentile at the 
beginning of the school year or the end of the previous school year. 

Type of intervention. Studies had to include certain types of interventions; other types of 
interventions were excluded. 

• The review included studies of the following interventions: 

o Interventions that were implemented for at least eight hours, 
o Interventions implemented during or after school or during the summer 
holiday. 
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O Interventions delivered by teachers, interventionists, researchers, tutors, 
parents, or paraprofessionals (as long as they followed a specific intervention 
program or an approach that was clearly spelled out). 

• The review excluded studies of the following interventions: 

o Whole-class interventions or differentiated instruction (tier 1). 

° Whole-class interventions (tier 1) with subanalyses for at-risk readers. 

° Special education or tier 3 interventions, 
o At-home-only interventions. 

o Spanish interventions for at-risk or struggling readers. 

° Peer-tutoring interventions, 
o Professional development programs, 
o Phonemic awareness interventions. 

o Small-group interventions for low-income schools or students, 
o Computer-based interventions (if there was no interventionist). 

Outcomes. The study had to include at least one student-level outcome that demonstrat- 
ed sufficient reliability and content validity. Relevant outcomes were measures of student 
achievement, including nationally normed tests, other standardized tests, and researcher- 
developed measures, in the following areas: 

• Word and pseudoword reading. This area included both timed and untimed mea- 
sures. Possible measures included: 

° Word lists, such as Test of Word Reading Efficiency and Woodcock Letter- 
Word ID. 

° Pseudowords, such as Dynamic Indicators of Basic Early Literacy Skills Non- 
sense Word Fluency and Woodcock Johnson Word Attack. 

• Passage reading fluency. This area included only timed measures. Possible measures 
include AIMSweb Standard Reading Assessment Passages. 

• Reading comprehension. This area included both measures with questions and 
those that use cloze/maze procedures. Possible measures included the Woodcock 
Reading Mastery Tests Passage Comprehension subtest and the Group Reading 
Assessment and Diagnostic Evaluation reading comprehension subtest. 

• Reading vocabulary. This area included reading vocabulary, receptive vocabulary, 
and academic vocabulary. 7 Possible measures included the Group Reading Assess- 
ment and Diagnostic Evaluation reading vocabulary subtest and the Ginn Total 
Vocabulary. 

• Overall reading achievement. This area’s outcomes include total scores on standard- 
ized reading tests. Possible measures included the Gates-MacGinitie Reading Test 
and the Stanford Achievement Test. 8 

This process screened out 1,770 studies, leaving 43 to be reviewed by a WWC-certified 
reviewer. Any study previously reviewed by the WWC that met standards (with or without 
reservations) was reviewed again using the current WWC standards (Version 3.0). 

Reviewing studies using What Works Clearinghouse standards 

Studies were reviewed using the WWC Procedures and Standards Handbook, Version 3.0 
(What Works Clearinghouse, 2014a) for group design studies. Although a WWC-certified 
reviewer reviewed each study that met the screening criteria and the review is modeled 
after the WWC approach to reviewing causal evidence, this report is not a WWC product. 
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A second reviewer independently reviewed studies that the first reviewer determined to 
have met standards. A senior reviewer double-checked each completed review to ensure 
accuracy and reconciled any differences between the reviewers. The summary of interverv 
tions and the evidence described in this review are limited to studies that the reviewers 
determined met WWC evidence standards with or without reservations. 

Of the 43 studies reviewed, 16 were rated as does not meet evidence standards (table Al). 
Ten of the 16 (63 percent) were not able to demonstrate baseline equivalence for the 
treatment and comparison groups used for the statistical analysis, 5 of the 16 (31 percent) 
could not attribute the measure of effectiveness solely to the intervention, and 1 of the 16 
(6 percent) did not include a valid and reliable outcome. 

The remaining 27 studies that met WWC standards as determined by the review team are 
described in detail in appendix C. 

Methods of calculating effect sizes 

The review team analyzed the studies that met WWC standards in three ways: 

1. To determine the impact of each intervention, the review team calculated the average 
effect size (and its statistical significance) for each area of reading (that is, word and 
pseudoword reading, passage reading fluency, reading comprehension, and vocabulary). 
• For each continuous outcome, the review team calculated Hedges’ g using the 
WWC Study Review Guide (What Works Clearinghouse, 2014b). Hedges’ g is the 
difference between the mean outcome of the intervention group and the mean 
outcome of the comparison group divided by the pooled withiivgroup standard 
deviation of the outcome measure. Impacts from dichotomous outcomes were 
computed using the log odds ratio (see What Works Clearinghouse, 2014a, for 
additional information). 


Table Al. Reasons studies were rated as does not meet What Works Clearinghouse 
standards, as determined by the review team 


Reason 

Number of studies 
with rating 

Share of total (percent) 

Baseline equivalence of the treatment and comparison 
groups used for the analysis was not demonstrated 3 

10 

63 

The measure of effectiveness could not be attributed 

solely to the intervention 

5 

31 

The eligible outcomes did not meet What Works 
Clearinghouse requirements for validity and reliability 13 

1 

6 

Total 

16 

100 


a. According to What Works Clearinghouse standards for group design, baseline equivalence should be 
demonstrated based on a conservative boundary for quasi-experimental design studies as well as randomized 
controlled trials with high attrition or randomization problems. 

b. According to What Works Clearinghouse standards for group design, outcome measures must demonstrate 
sufficient reliability and content validity. Reliability for group-design studies was assessed using the following 
standards determined by the What Works Clearinghouse: internal consistency (minimum of .50), temporal 
stability or test-retest reliability (minimum of .40), or interrater reliability (minimum of .50). 

Source: Authors' analysis of primary data collected for the review. 


A-4 




• If a study had multiple outcomes within a reading area (that is, word and pseudo- 
word reading, passage reading fluency, reading comprehension, and vocabulary), 
the effect sizes for all of that study’s outcomes were combined into a study average 
effect size using the simple, unweighted average of the individual effect sizes. 
Simple averages were used because the sample sizes of each outcome measure for a 
particular area of reading within a particular study are identical or virtually iden- 
tical. These calculations were done using all eligible effect sizes (significant and 
nonsignificant) in the WWC Study Review Guide. According to the Study Review 
Guide instructions for group design studies, domain average effect sizes are calcu- 
lated in column C, which is “a locked cell with a formula to calculate the average 
effect size for all outcomes in the domain entered in Column A” (What Works 
Clearinghouse, 2014b, p. 38). 

2. Author-reported p-values were used unless otherwise noted for individual outcome 
measures. In certain circumstances, p-values calculated using the WWC Study Review 
Guide were used for individual outcome measures. The Study Review Guide also calcu- 
lated the p-values for the average effect sizes for each area of reading. According to the 
Study Review Guide instructions for group design studies, “This is calculated based on 
the hidden t-statistic (Column N) which is based on the average effect size (Column 
C)” (What Works Clearinghouse, 2014b, p. 38). To gauge the impact of reading inter- 
ventions across all the interventions for students in grade 1 and in grades 2 and 3, the 
review team also calculated weighted mean effect sizes for each area of reading. Each 
intervention was weighted in inverse proportion to its variance, which typically gives 
more weight to interventions with large sample sizes and less weight to interventions 
with small sample sizes. Thus, results from interventions with large samples will be less 
affected by sampling error and therefore are given more power in the analysis. 

3. The review team examined the description of the intervention and cataloged the 
intervention characteristics to look for patterns that might explain the impacts. 

Interventions with studies that met standards were categorized using their average effect 
in an area of reading (see box 1 in main text). Interventions were found to demonstrate 
a statistically significant average effect in any of the four areas of reading, demonstrate a 
potentially positive average effect size greater than 0.25 that was not statistically signifi- 
cant, or fail to produce such an effect. 


Appendix B. Forty-three studies reviewed 
using What Works Clearinghouse standards 


Of the 1,813 studies identified in the literature search, the 43 studies listed in this appendix 
used an experimental or quasi-experimental design to examine the impact of interventions 
intended to improve reading outcomes for students who are at risk of reading difficulties 
in grades 1-3, were published since the adoption of No Child Left Behind in 2002, and 
were conducted in the United States. The review team determined that 27 of those met 
the What Works Clearinghouse standards (Version 3.0; 2014) either with or without reser- 
vations; an asterisk identifies these studies. Vadasy et al. (2006) and Allor and McCathren 
(2004) included two studies each; therefore, only 25 citations are identified with an aster- 
isk. Although this review relies heavily on WWC protocols, procedures, and standards, 
and WWC-certified reviewers conducted the reviews, this review is not a WWC product. 

*Allor, J., & McCathren, R. (2004). The efficacy of an early literacy tutoring program 
implemented by college students. Learning Disabilities Research & Practice, 19(2), 116— 
129. http://eric.ed.gov/lkUEJ687003 

*Berninger, V. W., Abbott, R. D., Vermeulen, K., & Fulton, C. M. (2006). Paths to reading 
comprehension in at-risk second-grade readers. Journal of Learning Disabilities, 39(4), 
334-351. http://eric.ed.gov/lkUEJ757962 

*Blachman, B. A., Schatschneider, C., Fletcher, J. M., Francis, D. J., Clonan, S. M., 
Shaywitz, B. A., & Shaywitz, S. E. (2004). Effects of intensive reading remediation 
for second and third graders and a 1-year follow-up. Journal of Educational Psychology, 
96(3), 444-461. http://eric.ed.gov/lkUEJ685003 

Brown, K. J., Morris, D., & Fields, M. (2005). Intervention after grade 1: Serving increased 
numbers of struggling readers effectively. Journal of Literacy Research, 37(1), 61—94- 
http://eric.ed.gov/lkUEJ728578 

*Case, L. P., Speece, D. L., Silverman, R., Ritchey, K. D., Schatschneider, C., Cooper, D. H., 
& Jacobs, D. (2010). Validation of a supplemental reading intervention for first-grade 
children. Journal of Learning Disabilities, 43(5), 402-417. http://eric.ed.gov/lkUEJ896983 

*Case, L. P., Speece, D., Silverman, R., Schatschneider, C., Montanaro, E., & Ritchey, 
K. (2014). Immediate and long-term effects of tier 2 reading instruction for first-grade 
students with a high probability of reading failure. Journal of Research on Educational 
Effectiveness, 7(1), 28-53. http://eric.ed.gov/lkUEJ1030356 

*Denton, C. A., Cirino, P. T., Barth, A. E., Romain, M., Vaughn, S., Wexler, J., & Fletcher, 
J. M. (2011). An experimental study of scheduling and duration of “tier 2” first-grade 
reading intervention. Journal of Research on Educational Effectiveness, 4(3), 208-230. 
http://eric.ed.gov/lkUEJ932550 

*Denton, C. A., Fletcher, J. M., Taylor, W. P, Barth, A. E., & Vaughn, S. (2014). An exper- 
imental evaluation of guided reading and explicit interventions for primary-grade stu- 
dents at-risk for reading difficulties. Journal of Research on Educational Effectiveness, 
7(3), 268-293. http://eric.ed.gov/lkUEJ1032919 
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*Denton, C. A., Nimon, K., Mathes, P. G., Swanson, E. A., Kethley, C., Kurz, T. B., & 
Shih, M. (2010). Effectiveness of a supplemental early reading intervention scaled up in 
multiple schools. Exceptional Children, 76(4), 394-416. http://eric.ed.gov/lkHEJ891824 

Fien, EL, Smith, J. L. M., Smolkowski, K., Baker, S. K., Nelson, N. J., & Chaparro, E. 
(2014). An examination of the efficacy of a multitiered intervention on early reading 
outcomes for first grade students at risk for reading difficulties. Journal of Learning Dis- 
abilities, 48(6), 602-621. 

*Fuchs, D., Compton, D. L., Fuchs, L. S., Bryant, J., & Davis, G. N. (2008). Making 
“secondary intervention” work in a three-tier responsiveness-to-intervention model: 
Findings from the first-grade longitudinal reading study of the National Research 
Center on Fearning Disabilities. Reading and Writing, 21(4), 413-436. http://eric.ed. 
gov/?id=EJ793405 

Gattis, M. N., Morrow-Howell, N., McCrary, S., Fee, M., Jonson-Reid, M., McCoy, EL, 
& Invernizzi, M. (2010). Examining the effects of New York Experience Corps® 
Program on young readers. Literacy Research and Instruction, 49(4), 299-314. http://eric. 
ed.gov/?id=EJ897919 

*Gunn, B., Smolkowski, K., Biglan, A., Black, C., & Blair, J. (2005). Fostering the 
development of reading skill through supplemental instruction: Results for Hispan- 
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Appendix C. Research basis for the studies that the review team 
determined met What Works Clearinghouse evidence standards 


Tables C1-C4 in this appendix report the research design, analysis sample, intervem 
tion implementation, nature of the comparison group, outcome measures, and effect size 
estimates (Hedges’ g) for the studies that the review team determined met What Works 
Clearinghouse evidence standards with or without reservations. The tables are organized 
by grade level and grouping arrangements (that is, small group or individual). 
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Study design 
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sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Individual grouping 

Allor, J., & McCathren, 
R. (2004). The 
efficacy of an early 
literacy tutoring 

Randomized 

controlled trial 

Study 1: 

86 grade 1 
students in 8 

schools 

Early Literacy 

Tutoring 

Program 

No additional 

support in 
reading 

Reading 

comprehension 

• WJ-R Passage Comprehension 3 (0.50*) 

0.50* 

program implemented 


Study 2: 

Early Literacy 

No additional 

Word and 

• WJ-R Word Identification 3 (0.13) 

0.35* 

by college students. 
Learning Disabilities 
Research & Practice, 


157 grade 1 

Tutoring 

support in 

pseudoword 

• WJ-R Word Attack 3 (0.78***) 



students in 10 

schools 

Program 

reading 

reading 

TOWRE Sight Word Efficiency 3 (0.05) 


19(2), 116-129. 





TOWRE Phonemic Decoding Efficiency 3 (0.44**) 








- DIBELS Nonsense Word Fluency 3 (0.33) 







Reading 

comprehension 

• WJ-R Passage Comprehension 3 (-0.16) 

-0.16 






Passage 
reading fluency 

• Oral Reading Fluency 3 (0.13) 

0.13 

Jenkins, J. R., Peyton, 

Quasi- 

99 grade 1 

Researcher- 

No additional 

Word and 

• WRMT-R Word Attack 3 (0.76**) 

0.67* 

J. A., Sanders, E. 

experimental 

students in 

modified 

support in 

pseudoword 

• WRAT-R Reading 3 (0.73**) 


A., & Vadasy, P. F. 
(2004). Effects of 
reading decodable 
texts in supplemental 

design 

11 schools in 

1 district 

Sound 

Partners 

reading 

reading 

■ Bryant Diagnostic Test of Basic Decoding Skills 9 
(1.12***) 







• TOWRE Phonemic Decoding Efficiency 3 (0.37) 


first-grade tutoring. 






• WRMT-R Word Identification 3 (0.50) 


Scientific Studies of 






TOWRE Sight Word Efficiency 3 (0.52) 


Reading, 8(1), 53-85. 





Reading 

• WRMT-R Passage Comprehension 3 (0.74**) 

0.74** 


comprehension 
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Analysis 
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name group 
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performance 

area 
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Average 
effect size 
for reading 
area 


Lane, H. B., Pullen, Randomized 

P. C., Hudson, R.F., controlled trial 

& Konold, T. R. 

(2009). Identifying 
essential instructional 
components of 
literacy tutoring for 
struggling beginning 
readers. Literacy 
Research and 
Instruction, 48(4), 

277-297. 


41 grade 1 University of No additional Word and 
students in 12 Florida Literacy support in pseudoword 
schools in 1 Initiative reading reading 

district 


42 grade 1 University of No additional 

students in 12 Florida Literacy support in 
schools in 1 Initiative minus reading 
district manipulative 

letter 

component 

43 grade 1 University of No additional 

students in 12 Florida Literacy support in 
schools in 1 Initiative minus reading 
district sentence 

writing strategy 


Word and 

pseudoword 

reading 


Word and 

pseudoword 

reading 


• Nonword decoding (0.64*) 

• Sight words 8 (0.71*) 

• WDRB Word Attack 8 (1.24***) 


• Nonword decoding (0.29) 
■ Sight words (0.24) 


• Nonword decoding (0.55) 
■ Sight words (0.39) 


0 . 86 * 


0.26 


0.47 


46 grade 1 University of No additional 

students in 12 Florida Literacy support in 

schools in 1 Initiative minus reading 

district extending 

literacy 
component 


Word and 

pseudoword 

reading 


• Nonword decoding (0.59) 

• Sight words (0.52) 

• WDRB Word Attack (1.02**) 


0.71* 


No additional Word and 
support in pseudoword 

reading reading 

model on the early 
literacy development 
of struggling first 
grade students. 

Literacy Research 
and instruction, 43(4), 

21-40. 


Pullen, P. C., Lane, H. 

B. , & Monaghan, M. 

C. (2004). Effects of 
a volunteer tutoring 


Randomized 
controlled trial 


47 grade 1 Volunteer 
students in 10 tutoring 
schools in 1 
district 


• Jump Start in Reading Assessment of Early Literacy 0.55 

Development: Sight Words 8 (0.24) 

• Jump Start in Reading Assessment of Early Literacy 
Development: Nonword Decoding 8 (0.81**) 

• WDRB Word Attack subtest (0.59*) 

• WDRB Letter-Word ID subtest (0.54*) 


(continued) 




C-4 


Table Cl. Impact of grade 1 reading interventions, by study (continued) 







Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Scanlon, D. M., 

Randomized 

114 grade 1 

Intensive 

Typical 

Word and 

• WRMT-R Word Identification 3 (0.55**) 

0.43* 

Vellutino, F. R., 

Small, S. G., Fanuele, 

controlled trial 

students in 5 

districts 

Tutorial 

Intervention 

school- 

provided 

pseudoword 

reading 

• WRMT-R Word Attack 3 (0.31) 


D. P., & Sweeney, 

J. M. (2005). 



— Text 

Emphasis 

reading 

intervention 

Reading 

comprehension 

• WIAT Reading Comprehension 3 (0.41*) 

0.41* 

Severe reading 

difficulties — can 

they be prevented? 

A comparison 
of prevention 

and intervention 

approaches. 
Exceptionality, 13(4), 

209-227. 


117 grade 1 

Intensive 

Typical 

Word and 

• WRMT-R Word Identification 3 (0.51**) 

0.57*** 


students in 5 

districts 

Tutorial 

Intervention — 

school- 

provided 

pseudoword 

reading 

• WRMT-R Word Attack 3 (0.62**) 




Phonological 

Skills 

Emphasis 

reading 

intervention 

Reading 

comprehension 

• WIAT Reading Comprehension 3 (0.35) 

0.35 

Schwartz, R. M. 

Randomized 

74 grade 1 

Reading 

No additional 

Word and 

• Ohio Word Test 3 (1.37***) 

1.15*** 

(2005). Literacy 
learning of at-risk 

controlled 

trial b 

students 

Recovery 

support in 
reading 

pseudoword 

reading 

• Slosson Oral Reading Test — Revised 3 (0.93***) 


first-grade students in 
the Reading Recovery 
early intervention. 

Journal of Educational 

Psychology, 97(2), 
257-267. 





Reading 

comprehension 

• Degrees of Reading Power Test 3 (0.14) 

0.14 

Vadasy, P. F., & 
Sanders, E. A. 

(2011). Efficacy 

Randomized 

controlled trial 

89 grade 1 
students in 

11 schools in 

Phonics- 
based Early 
Reading 

No additional 
support in 
reading 

Word and 

pseudoword 

reading 

• WRMT-R/NU Word Reading (Word Attack & Word 
Identification) 3 (0.51*) 

0.51* 

of supplemental 
phonics-based 


1 district 

Intervention 


Reading 

comprehension 

• WRMT-R/NU Passage Comprehension 3 (0.29) 

0.29 

instruction for low- 

skilled first graders: 
How language minority 
status and pretest 

characteristics 

moderate treatment 

response. Scientific 
Studies of Reading, 
15(6), 471-497. 





Passage 
reading fluency 

• Passage reading fluency 3 (0.69**) 

0.69** 
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Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Vellutino, F. R., 

& Scanlon, D. 

M. (2002). The 
Interactive Strategies 
approach to reading 

intervention. 

Contemporary 

Educational 

Psychology, 27(4), 

573-635. 

Randomized 

controlled trial 

118 grade 

1 and 2 

students in 17 

schools in 5 

districts 0 

Interactive 

Strategies 

Typical 

school- 

provided 

reading 

intervention 

Word and 

pseudoword 
reading (above 

the 15th 

percentile) 

• WRMT-R Basic Skills Cluster 3 (0.38*) 

0.38* 

Small group 

Case, L. P., Speece, 

Randomized 

30 grade 1 

Combined 

No additional 

Word and 

• WRMT-R/NU Word Attack 3 (0.73) 

0.65 

D. L., Silverman, 

controlled trial 

students in 

Fundations, 

support in 

pseudoword 

• WRMT-R/NU Word Identification 3 (0.73) 


R., Ritchey, K. D., 
Schatschneider, C., 
Cooper, D. H., Jacobs, 
D. (2010). Validation 
of a supplemental 
reading intervention 
for first-grade 

children. Journal of 

Learning Disabilities, 
43(5), 402-417 


3 schools in 

1 district 

Responsive 

Reading 

Instruction, 

and Read 

Naturally 

reading 

reading 

• Word Identification Fluency 3 (0.48) 


Case, L. P., Speece, 

Randomized 

123 grade 1 

Combined 

No additional 

Word and 

• Word Identification Fluency 3 (0.17) 

0.12 

D. L., Silverman, R., 

controlled trial 

students 

Fundations, 

support in 

pseudoword 

• WRMT-R/NU Word Attack 3 (0.23) 


Schatschneider, C., 
Montanaro, E., & 
Ritchey, K. D. (2014). 



Responsive 

Reading 

Instruction, 

reading 

reading 

TOWRE Phonemic Decoding Efficiency 3 (0.02) 






• WRMT-R/NU Word Identification 3 (0.21) 


Immediate and long- 



and Read 



• Graphophonemic fluency 3 (0.00) 


term effects of tier 2 

reading instruction for 
first-grade students 
with a high probability 
of reading failure. 

Journal of Research 

on Educational 

Effectiveness, 7(1), 

28-53. 



Naturally 


Passage 
reading fluency 

• Passage Reading Fluency 3 (0.20) 

0.20 
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Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Denton, C. A., Nimon, 
K., Mathes, P. G., 
Swanson, E. A., 

Randomized 

controlled 

trial e 

422 grade 1 

students in 31 

schools in 16 

Responsive 

Reading 

Instruction 

Typical 

school- 

provided 

Word and 

pseudoword 

reading 

TOWRE Phonemic Decoding Efficiency 3 (0.42***) 

q 42*** 

Kethley, C., Kurz, T. 

B., & Shih, M. (2010). 

Effectiveness of a 

supplemental early 
reading intervention 
scaled up in multiple 
schools. Exceptional 
Children, 76(4), 

394-416. 


districts 


reading 

intervention 

Reading 

comprehension 

• WJ-III Passage Comprehension 3 (0.51***) 

0.51*** 

Fuchs, D., Compton, 

Cluster 

64 grade 1 

Reading 

No additional 

Word and 

• TOWRE Sight Word Efficiency 3 (0.65) 

0.40 

D. L., Fuchs, L. 

randomized 

students in 

intervention 

support in 

pseudoword 

• Word Identification Fluency 3 (0.46) 


S., Bryant, J., & 

Davis, G. N. (2008). 
Making “secondary 

controlled trial 

16 schools in 

2 districts 

developed by 

Fuchs et al. 

reading 

reading 

• WRMT-R/NU Word Identification 3 (0.26) 





• WRMT-R/NU Word Attack 3 (0.38) 


intervention” work 

in a three-tier 

responsiveness-to- 

intervention model: 

findings from the 
first-grade longitudinal 
reading study of the 

National Research 

Center on Learning 
Disabilities. Reading 
and Writing, 21(4), 
413-436. 






TOWRE Phonemic Decoding Efficiency 3 (0.26) 


Wang, C., & Algozzine, 

Cluster 

139 grade 1 

BRIC 

No additional 

Word and 

• WRMT-R Word Attack 3 (0.45) 

0.27 

B. (2008). Effects of 

randomized 

students in 6 

Supplemental 

support in 

pseudoword 

• WRMT-R Word Identification 3 (0.39) 


targeted intervention 
on early literacy skills 

of at-risk students. 

Journal of Research in 

Childhood Education, 
22(4), 425-439. 

controlled 

trial d 

schools in 1 

district 

Intervention 

reading 

reading 

• DIBELS Nonsense Word Fluency 3 (-0.03) 




Reading 

comprehension 

• WRMT-R Passage Comprehension 3 (0.17) 

0.17 
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Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Findings 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Wanzek, J., & 

Quasi- 

50 grade 1 

Reading 

Typical 

Word and 

• WRMT-R Word Identification 3 (0.12) 

0.15 

Vaughn, S. (2008). 

experimental 

students in 6 

intervention 

school- 

pseudoword 

• WRMT-R Word Attack 3 (0.18) 


Response to varying 

design 

schools in 1 

developed 

provided 

reading 



amounts of time in 


district 

by Wanzek & 

reading 

Passage 

• DIBELS Oral Reading Fluency 3 (-0.20) 

-0.20 

reading intervention 



Vaughn 

intervention 

reading fluency 




for students with 
low response 
to intervention. 

Journal of Learning 
Disabilities, 41(2), 

126-142. 

* Significant at p < .05; * ** significant at p < .01; *** significant at p < .001. 

DIBELS is Dynamic Indicators of Basic Early Literacy Skills. TOWRE is Test of Word Reading Efficiency. WDRB is Woodcock Diagnostic Reading Battery. WIAT is Wechsler Individual 
Achievement Test. WJ-III is Woodcock Johnson III. WJ-R is Woodcock Johnson, Revised, Tests of Achievement. WRAT-R is Wide Range Achievement Test, Revised. WRMT-R/NU is 
Woodcock Reading Mastery Test, Revised/Normative Update. 

Note: Individual effect sizes are presented only for outcomes that were eligible or met standards. All individual effect size p-values are review team reported unless otherwise noted. 
Average effect sizes for reading area include significant and nonsignificant outcomes and were calculated by the review team. 

a. WWC-computed statistical significance. 

b. Randomized controlled trial; however, attrition information was not available. Baseline equivalence was established. 

c. Some grade 1 students continued to receive the intervention after they transitioned to grade 2. 

d. Randomization was compromised. Clusters had nonequal random assignment. 

e. Randomization was compromised. Nonequal random assignment of students. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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Table C2. Impact of grade 2 and 3 reading interventions, by study 







Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Individual grouping 

Blachman, B. A., 

Randomized 

69 grade 

Reading 

Typical school- 

Word and 

• WRMT-R Word Attack (0.74***) 

0.80*** 

Schatschneider, C., 

controlled 

2 and 3 

intervention 

provided reading 

pseudoword 

■ WRMT-R Word ID (0.87***) 


Fletcher, J. M., 

trial 3 

students in 

developed 

intervention 

reading 



Francis, D. J., 

Clonan, S. M., 


11 schools in 

4 districts 

by Blachman 

et al. 


Reading 

comprehension 

■ G0RT-3 Comprehension (0.53*) 

0.53* 

Shaywitz, B. A., 

& Shaywitz, S. E. 

(2004). Effects of 





Passage 
reading fluency 

■ G0RT-3 Quotient (0.70**) 

0.70** 


intensive reading 
remediation for 
second and third 
graders and a 1-year 
follow-up. Journal 
of Educational 


Psychology, 96(3), 
444-461. 


O'Connor, R. E., 

Randomized 40 grade 2 

Reading to 

No additional 

Word and 

■ WRMT-R/NU Word Identification 3 (0.56) 

0.33 

Swanson, H. L., & 

controlled trial students in 5 

an adult 

support in 

pseudoword 

• WRMT-R/NU Word Attack 3 (0.10) 


Geraghty, C. (2010). 

schools 

listener — 

reading 

reading 



Improvement in 
reading rate under 


Independent 

Level 


Reading 

comprehension 

• WRMT-R/NU Passage Comprehension 3 (0.48) 

■ GORT-4 Comprehension 3 (0.53) 

0.51 

independent and 

difficult text levels: 

Influences on word 

and comprehension 

skills. Journal 

of Educational 




Passage 

■ GORT-4 Fluency 3 (0.60) 

0.75* 




reading fluency 

• Grade 2 passages 3 (0.76*) 

• Primer-level passage 3 (0.87**) 

• Grade 1 passage 3 (0.75*) 





Vocabulary 

• WRMT-R/NU Vocabulary 3 (-0.12) 

-0.12 

Psychology, 102(1), 

43 grade 2 

Reading to an 

No additional 

Word and 

• WRMT-R/NU Word Identification 3 (0.57) 

0.41 

1-19. 

students in 5 

adult listener 

support in 

pseudoword 

• WRMT-R/NU Word Attack 3 (0.25) 



schools 

— Difficult 

reading 

reading 





Level 


Reading 

• GORT-4 Comprehension 3 (0.44) 

0.40 





comprehension 

• WRMT-R/NU Passage Comprehension 3 (0.37) 






Passage 

• GORT-4 Fluency 3 (0.81*) 

0.98*** 





reading fluency 

• Grade 2 passages 3 (0.84**) 

• Primer-level passage 3 (1.33***) 

• Grade 1 passage 3 (0.93**) 






Vocabulary 

■ WRMT-R/NU Vocabulary 3 (-0.04) 

-0.04 

(continued) 
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Findings 


Table C2. Impact of grade 2 and 3 reading interventions, by study (continued) 


Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Vadasy, P. F., 

Sanders, E. A., & 
Peyton, J. A. (2006). 

Paraeducator- 

supplemented 

instruction in 

Quasi- 

experimental 

design 

31 grade 

2 and 3 

students in 

12 schools in 

1 district 

Paraeducator- 

supplemented 

instruction 

No additional 

support in 
reading 

Word and 

pseudoword 

reading 

• Reading accuracy composite measure: WRAT 
Reading, WRMT-R/NU Word Identification, WRMT-R/ 
NU Word Attack b (0.61) 

Reading efficiency composite measure: TOWRE 
Phonemic Decoding Efficiency, TOWRE Sight Word 
Efficiency 11 (0.72) 

0.67 

structural analysis 
with text reading 





Reading 

comprehension 

■ WRMT-R/NU Passage Comprehension 1 " (0.50) 

0.50 

practice for second 
and third graders 
at risk for reading 
problems. Remedial 
and Special 

Education, 2/(6), 
365-378. 





Passage 
reading fluency 

■ Grade-level passages 1 " (0.81) 

0.81 

Randomized 

controlled trial 

21 grade 

2 and 3 

students in 5 

schools 

Paraeducator- 

supplemented 

instruction 

No additional 

support in 
reading 

Word and 

pseudoword 

reading 

Reading accuracy composite measure: WRAT 
Reading, WRMT-R/NU Word Identification, WRMT-R/ 
NU Word Attack 1 " (0.75) 

Reading efficiency composite measure: TOWRE 
Phonemic Decoding Efficiency, TOWRE Sight Word 
Efficiency 1 " (0.67) 

0.71 






Reading 

comprehension 

• WRMT-R/NU Passage Comprehension 1 " (0.21) 

0.21 






Passage 
reading fluency 

■ Grade-level passages 1 " (0.55) 

0.55 

Vadasy, P. F., 

Sanders, E. A., & 
Tudor, S. (2007). 

Randomized 

controlled trial 

43 grade 

2 and 3 

students in 9 

Paraeducator- 

supplemented 

instruction 

No additional 
support in 
reading 

Word and 

pseudoword 

reading 

■ WRMT-R/NU Word Identification and Word Attack 
composite 1 " (0.47) 

0.47 

Effectiveness of 

paraeducator- 


schools in 1 

district 



Passage 
reading fluency 

• Grade-level passages from DIBELS (0.52) b 

0.52 


supplemented 

individual 

instruction: Beyond 
basic decoding 
skills. Journal of 
Learning Disabilities, 
40(6), 508-525. 


(continued) 
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Table C2. Impact of grade 2 and 3 reading interventions, by study (continued) 







Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention 

name 

Nature of the 

comparison 

group 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Small grouping 

Berninger, V. W., 
Abbott, R. D., 
Vermeulen, K., & 
Fulton, C. M. (2006). 
Paths to reading 
comprehension in 
at-risk second-grade 

readers. Journal of 

Learning Disabilities, 
39(4), 334-351. 

Randomized 

controlled trial 

93 grade 2 

students in 8 

schools in 1 

district 

Before/ 

After School 

Reading Club 

No additional 

support in 
reading 

Word and 

pseudoword 

reading 

• WRMT-R Word Attack b (0.35) 

0.35 

Denton, C. A., 
Fletcher, J. M., 

Taylor, W. P., 

Barth, A. E., & 

Randomized 

controlled trial 

103 grade 

1 and 2 

students in 9 

schools in 2 

Guided 

Reading 

No additional 
support in 
reading 

Word and 

pseudoword 

reading 

• WJ -III Letter-Word Identification (0.50*) 

• WJ III Word Attack (0.34) 

• Composite: TOWRE Phonemic Decoding Efficiency, 
TOWRE Sight Word Efficiency (0.40) 

0.41* 

Vaughn, S. (2014). 

An experimental 


districts 0 



Reading 

comprehension 

■ WJ -III Passage Comprehension (0.13) 

■ Gates-MacGinitie Reading Comprehension (0.08) 

0.11 

evaluation of guided 
reading and explicit 

interventions for 

primary-grade 

students at-risk for 

reading difficulties. 

Journal of Research 





Passage 
reading fluency 

• TPRI progress monitoring for beginning readers 
(0.16) 

0.16 


112 grade 

1 and 2 

students in 9 

schools in 2 

Explicit, 

sequential 

instruction 

in word 

No additional 

support in 
reading 

Word and 

pseudoword 

reading 

■ WJ III Word Attack (0.63**) 

■ WJ -III Letter-Word Identification (0.50*) 

• Composite of TOWRE Phonemic Decoding Efficiency 
and TOWRE Sight Word Efficiency (0.31) 

0.48* 

on Educational 

Effectiveness, 7(3), 


districts' 1 

reading and 
comprehension 


Reading 

comprehension 

■ Gates-MacGinitie Reading Comprehension (0.29) 

■ WJ -III Passage Comprehension (0.46*) 

0.37 

268-293. 





Passage 
reading fluency 

■ TPRI progress monitoring for beginning readers 
(0.45*) 

0.45* 


(continued) 
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Table C2. Impact of grade 2 and 3 reading interventions, by study (continued) 








Findings 






Nature of the 

Reading 


Average 
effect size 



Analysis 

Intervention 

comparison 

performance 


for reading 

Full citation 

Study design 

sample 

name 

group 

area 

Individual measure (effect size) 

area 

Gunn, B., 

Randomized 

245 grade 

Reading 

No additional 

Word and 

• WJ-R Letter-Word Identification 11 (0.30*) 

0.41*** 

Smolkowski, K., 

controlled trial 

1-4 students 

Mastery/ 

support in 

pseudoword 

• WJ-R Word Attack (0.52***) 


Biglan, A., Black, 


in 13 schools 

Corrective 

reading 

reading 



C., & Blair, J. 

(2005). Fostering 


in 4 districts 8 

Reading 


Reading 

comprehension 

• WJ-R Passage Comprehension 8 (0.32*) 

0.32* 

the development of 
reading skill through 
supplemental 
instruction: Results 





Passage 
reading fluency 

• DIBELS Oral Reading Fluency 8 (0.24) 

0.24 





Vocabulary 

• WJ-R Vocabulary 8 (0.22) 

0.22 


for Hispanic and non- 
Hispanic students. 

Journal of Special 
Education, 39(2), 

66-85 

* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

DIBELS is Dynamic Indicators of Basic Early Literacy Skills. GORT-3 is Gray Oral Reading Tests, 3rd edition. GORT-4 is Gray Oral Reading Tests, 4th edition. TOWRE is Test of Word 
Reading Efficiency. TPRI is Texas Primary Reading Inventory (Foorman et al., 2004). WJ-III is Woodcock Johnson III. WJ-R is Woodcock-Johnson Revised Tests of Achievement. WRAT 
is Wide Range Achievement Test. WRMT-R/NU is Woodcock Reading Mastery Test, Revised. 

Note: Individual effect sizes are presented only for outcomes that were eligible or met standards. All individual effect size p-values are review team reported unless otherwise noted. 
Average effect sizes for reading area include significant and nonsignificant outcomes and were calculated by the review team. 

a. Randomization was compromised. Nonequal random assignment of students. 

b. What Works Clearinghouse-computed statistical significance. 

c. Some students who were repeating grade 1 were included (15 percent of the sample). 

d. Some students who were repeating grade 1 were included (13 percent of the sample). 

e. The intervention was conducted over two years for students in grades 1-4. At baseline, 54 percent of the sample consisted of students in grades 2 and 3. Results reported are 
for the combined sample of Hispanic and non-Hispanic students. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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Table C3. Impact of studies with variations in aspects of treatment 








Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention A 

Intervention B 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Modifications 

Denton, C. A., Cirino, 

Randomized 

130 grade 1 

Read Well 

Read Well 

Word and 

• WJ-III Letter Word ID b (-0.15) 

-0.14 

P. T„ Barth, A. E., 

controlled 

students in 9 

Extended 

Concentrated 

pseudoword 

• WJ-III Word Attack b (-0.10) 


Romain, M., Vaughn, 

trial 3 

schools in 2 

Group 

Group 

reading 

T0WRE b (-0.17) 


S„ Wexler, J., & 


districts 



Reading 

• WJ-III Passage Comprehension 11 (-0.25) 

-0.05 

Fletcher, J. M. (2011). 





comprehension 

• GRADE Passage Comprehension 11 (0.15) 


An experimental 
study of scheduling 

and duration of 

"tier 2" first-grade 
reading intervention. 





Passage 
reading fluency 

• Continuous Monitoring of Early Reading Skills 11 
(-0.07) 

-0.07 


128 grade 1 

students in 9 

Read Well 

Extended 

Read Well 

Distributed 

Word and 

pseudoword 

• WJ-III Letter Word ID b (-0.04) 

• WJ-III Word Attack b (0.08) 

-0.01 

Journal of Research 


schools in 2 

Group 

Group 

reading 

T0WRE b (-0.07) 


on Educational 


districts 



Reading 

• WJ-III Passage Comprehension 11 (-0.13) 

-0.09 

Effectiveness, 4(3), 





comprehension 

• GRADE Passage Comprehension 11 (-0.06) 


208-230. 


126 grade 1 

Read Well 

Read Well 

Word and 

• WJ-III Letter Word ID b (0.12) 

0.13 



students in 9 

Concentrated 

Distributed 

pseudoword 

• WJ-III Word Attack b (0.18) 




schools in 2 

Group 

Group 

reading 

T0WRE b (0.10) 




districts 



Reading 

• WJ-III Passage Comprehension 11 (0.13) 

-0.04 






comprehension 

• GRADE Passage Comprehension 11 (-0.20) 







Passage 

• Continuous Monitoring of Early Reading 11 Skills 

-0.02 






reading fluency 

(-0.02) 


Jenkins, J. R., Peyton, 

Randomized 

79 grade 1 

Researcher- 

Researcher- 

Word and 

• WRMT-R Word Attack 11 (0.18) 

0.05 

J. A., Sanders, E. 

controlled trial 

students in 

modified 

modified 

pseudoword 

• WRAT-R Reading 11 (0.05) 


A., & Vadasy, P. F. 


11 schools 

Sound 

Sound 

reading 

• Bryant Diagnostic Test of Basic Decoding Skills 11 


(2004). Effects of 



Partners — 

Partners — 


(-0.12) 


reading decodable 



More 

Less 


• TOWRE Phonemic Decoding Efficiency 11 (0.00) 


texts in supplemental 



Decodable 

Decodable 


• WRMT-R Word Identification 11 (0.02) 


first-grade tutoring. 



Words 

Words 


TOWRE Sight Word Efficiency 11 (0.08) 


Scientific Studies of 






Text word list b (0.14) 


Reading, 8(1), 53-85. c 





Reading 

comprehension 

• WRMT-R Passage Comprehension 11 (0.10) 

0.10 






Passage 

• Phonetically controlled passages — accuracy 11 (0.18) 

0.06 






reading fluency 

• Nonphonetically controlled passages — accuracy 11 
(0.00) 

• Phonetically controlled passages — fluency (0.09) 

• Nonphonetically controlled passages — fluency 
(-0.01) 



(continued) 
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Table C3. Impact of studies with variations in aspects of treatment (continued) 








Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention A 

Intervention B 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

O'Connor, R. E., 

Randomized 

43 grade 2 

Practice 

Practice 

Word and 

• WRMT-R/NU Word Identification 11 (0.03) 

-0.07 

Swanson, H. L., & 

controlled trial 

students in 5 

reading 

reading aloud 

pseudoword 

• WRMT-R/NU Word Attack 11 (-0.16) 


Geraghty, C. (2010). 


schools 

aloud with 

with difficult 

reading 



Improvement in reading 
rate under independent 



independent 

levels of text 

levels of text 

(Difficult) 

Reading 

comprehension 

• WRMT-R/NU Passage Comprehension 11 (0.10) 

• GORT-4 Comprehension 11 (0.08) 

0.09 

and difficult text levels: 

Influences on word and 

comprehension skills. 
Journal of Educational 

Psychology, 102(1), 

1-19. d 



(Independent) 


Passage 

• GORT-4 Fluency 11 (0.21ns) 

0.12 





reading fluency 

• Grade 2 passages 11 (-0.03) 

• Primer-level passage 11 (0.16) 

• Grade 1 passage 11 (0.15) 






Vocabulary 

• WRMT-R/NU Vocabulary 11 (0.08) 

0.08 

Scanlon, D. M., 

Randomized 

113 grade 1 

Intensive 

Intensive 

Word and 

• WRMT-R Word Identification 11 (-0.10) 

0.08 

Vellutino, F. R., Small, 

controlled trial 

students in 5 

Tutorial 

Tutorial 

pseudoword 

• WRMT-R Word Attack 11 (0.25) 


S. G., Fanuele, D. 


districts 

Intervention- 

Intervention- 

reading 



P., & Sweeney, J. M. 
(2005). Severe reading 
difficulties — can 



Phonological 

Skills 

Emphasis 

Text Emphasis 
(TE) 

Reading 

comprehension 

• WIAT Reading Comprehension 11 (-0.08) 

-0.08 

they be prevented? 



(PSE) 






A comparison 
of prevention 
and intervention 
approaches. 
Exceptionality, 13(4), 
209-227. e 


Group size 


Schwartz, R. M., Randomized 

149 grade 1 

Reading 

Reading 

Word and 

• Ohio Word Test (0.43*) 

Schmitt, M. C., & Lose, controlled trial 

students 

Recovery 

Recovery 

pseudoword 

• Slosson Oral Reading Test — Revised (0.45*) 

M. K. (2012). Effects 



Small Groups 

reading 


of teacher-student 

ratio in response 

to intervention 

approaches. 

Elementary School 

Journal, 112(4), 



of 2, 3, or 5 




547-567. 


(continued) 
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Table C3. Impact of studies with variations in aspects of treatment (continued) 







Findings 


Full citation 

Study design 

Analysis 

sample 

Intervention A Intervention B 

Reading 

performance 

area 

Individual measure (effect size) 

Average 
effect size 
for reading 
area 

Vaughn, S., Linan- 

Quasi- 

26 grade 2 

1:1 1:3 

Word and 

• WRMT-R Word Attack b (0.41) 

0.41 

Thompson, S., 

experimental 

students in 


pseudoword 



Kouzekanani, K., 

design 

10 schools in 


reading 



Bryant, D. P., Dickson, 
S., & Blozis, S. A. 


2 districts 


Reading 

comprehension 

• WRMT-R Passage Comprehension 11 (0.42) 

0.42 

(2003). Reading 
instruction grouping for 
students with reading 
difficulties. Remedial 




Passage 
reading fluency 

T0RF b (0.03) 

0.03 


25 grade 2 

1:1 1:10 

Reading 

• WRMT-R Passage Comprehension 11 (0.78) 

0.78 

and Special Education, 


students in 


comprehension 



24(5), 301-315. 


10 schools in 

2 districts 


Passage 
reading fluency 

T0RF b (0.21) 

0.21 



27 grade 2 

1:3 1:10 

Reading 

• WRMT-R Passage Comprehension 11 (0.29) 

0.29 



students in 


comprehension 





10 schools in 

2 districts 


Passage 
reading fluency 

T0RF b (0.22) 

0.22 


* Significant at p < .05; * ** significant at p < .01; *** significant at p < .001. 

GORT-4 is Gray Oral Reading Tests, 4th Edition. GRADE is Group Reading Assessment and Diagnostic Evaluation. TORF is Test of Oral Reading Fluency. TOWRE is Test of Word Read- 
ing Efficiency. WDRB is Woodcock Diagnostic Reading Battery. WIAT is Wechsler Individual Achievement Test. WJ-III is Woodcock Johnson III. WRAT-R is Wide Range Achievement 
Test, Revised. WRMT-R is Woodcock Reading Mastery Test, Revised. WRMT-R/NU is Woodcock Reading Mastery Test, Revised/Normative Update. 

Note: Individual effect sizes are presented only for outcomes that were eligible or met standards. All individual effect size p-values are review team reported unless otherwise noted. 
Average effect sizes for reading area include significant and nonsignificant outcomes and were calculated by the review team. 

a. Randomization was compromised. Nonequal random assignment of students. 

b. What Works Clearinghouse-computed statistical significance. 

c. This study also included a comparison of the combined treatment groups with a control group and was displayed in table Cl. 

d. This study also included a comparison of each treatment group with a control group and was displayed in in table C2. 

e. This study also included a comparison of each treatment group with a control group and was displayed in table Cl. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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Table C4. Impact of studies that compare treatments 








Findings 









Average 
effect size 

Full citation 

Study design 

Analysis 

sample 

Intervention A 

Intervention B 

Reading area 

Individual measure (effect size) 

for reading 
area 

Denton, C. A., Fletcher, 

Randomized 

109 grades 

Guided 

Explicit, 

Reading 

• WJ-III Passage Comprehension (0.33) 

0.27 

J. M„ Taylor, W. 

controlled trial 

1 and 2 

Reading 

sequential 

comprehension 

• Gates-MacGinitie Reading Comprehension (0.21) 


P., Barth, A. E., & 
Vaughn, S. (2014). 

An experimental 


students in 9 

schools in 2 

districts 11 


instruction 

in word 

reading and 

Passage 
reading fluency 

TOSREC (0.12) 

0.12 

evaluation of guided 
reading and explicit 

interventions for 
primary-grade 

students at-risk for 

reading difficulties. 

Journal of Research 

on Educational 

Effectiveness, 7(3), 
268-293. a 




comprehension 




Mathes, P. G., Denton, 

Randomized 

163 grade 1 

Proactive 

Responsive 

Word and 

• WJ-III Letter Word ID (0.15) 

0.27 

C. A., Fletcher, J. 

controlled trial 

students in 6 

Reading 

Reading 

pseudoword 

• WJ-III Word Attack (0.38*) 


M., Anthony, J. L., 


schools in 1 



reading 




Francis, D. J., & 
Schatschneider, C. 
(2005). The effects of 
theoretically different 
instruction and student 
characteristics on the 
skills of struggling 
readers. Reading 
Research Quarterly, 
40(2), 148-182. 


district 


* Significant at p < .05. 

TOSREC is Test of Silent Reading Efficiency and Comprehension. WJ-III is Woodcock Johnson III. 

Note: Individual effect sizes are only presented for outcomes that were eligible or met standards. All individual effect size p-values are author reported unless otherwise noted. Aver- 
age effect sizes for reading area include significant and nonsignificant outcomes, and were calculated by the study team. 

a. This study also included a comparison of each treatment group with a control group, which was displayed in in table C2. 

b. Some students who were repeating grade 1 were included. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 




Appendix D. Summary of the weighted mean effect sizes by 
area of reading for grade 1 and grade 2 and 3 interventions 


Tables D1 and D2 in this appendix report the number of interventions, average sample size 
(across interventions), effect size estimates (Hedges’ g), standard errors, 95 percent confb 
dence intervals, and statistical significance for each of the four reading performance areas 
for all interventions that the review team determined met What Works Clearinghouse 
evidence standards with or without reservations. The tables are organized by grade level 
and grouping arrangements (that is, individual grouping or small group). 


Table Dl. Weighted mean effects of grade 1 interventions, by area of reading 



Number 

Average 



95 percent 
confidence 
interval 3 


hedges' g and 95 percent 









confidence interval 

performance 

ventions 

size 

e b 

error 

limit 

limit 

p-value -o .3 -o .2 -o.i o o.i 0.2 0.3 0.4 0.5 0.6 0.7 

Individual grouping 

Word and 
















, 



pseudoword reading 

8 

93 

0.533 

0.077 

0.381 

0.684 

<.001 









• 



Passage reading 



















fluency 

2 

123 

0.326 

0.129 

0.073 

0.578 

.012 












Reading 














, 





comprehension 

5 

94 

0.325 

0.096 

0.136 

0.514 

<.001 







■ 





Vocabulary 

0 

na 

na 

na 

na 

na 

na 












Small group 














Word and 














, 





pseudoword reading 

5 

150 

0.372 

0.077 

0.220 

0.523 

<.001 







■ 





Passage reading 



















fluency 

2 

87 

0.086 

0.153 

-0.213 

0.386 

.572 












Reading 















. 




comprehension 

2 

281 

0.437 

0.088 

0.263 

0.610 

<.001 








■ 




Vocabulary 

0 

na 

na 

na 

na 

na 

na 













na is not applicable. 

a. There is a 95 percent probability that the “true" effect size lies between the lower and upper limits. If the interval includes 0, the 
weighted mean effect size is not statistically significant. 

b. Weighted mean effect size. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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Table D2. Weighted mean effects of grade 2 and 3 interventions, by area of reading 



Number 

Average 


95 percent 
confidence 
interval 3 

Hedges’ g and 95 percent 

Area of reading 

of inter- 

sample 

Hedges’ Standard 

Lower Upper 

confidence interval 

performance 

ventions 

size 

g b error 

limit limit 

p-value -o .6 -o .4 -0.2 0 0.2 0.4 o.e 0.8 1.0 

Individual grouping 


Word and 

pseudoword reading 


47 0.629 0.174 0.287 0.971 <.001 


Passage reading 
fluency 


47 0.733 0.176 0.388 1.077 <.001 


Reading 

comprehension 


46 0.476 0.175 0.133 0.819 .006 


Vocabulary 


42 -0.080 0.311 -0.690 0.530 .797 



a. There is a 95 percent probability that the “true" effect size lies between the lower and upper limits. If the interval includes 
0, the weighted mean effect size is not statistically significant. 

b. Weighted mean effect size. 

Source: Authors' analysis of primary data collected for the review; see appendix A. 
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Notes 


The authors thank Samantha Spallone, Pam Foremski, and Christopher Tran for their 
assistance throughout the development of this report, as well as Madhavi Jayanthi for 
her guidance and input. The authors also acknowledge the efforts of the What Works 
Clearinghouse-certified reviewers who reviewed studies for this project: Michael Armijo, 
Gina Arnone, Scott Baker, Nicholas Gage, Abigail Gray, Constance Lindsay, Melinda 
Mollette, Nancy Nelson, Dan Player, and Erin Reid. 

1. Efficacy studies are tightly controlled studies during which fidelity of implementation 
is carefully monitored and interventionists receive high levels of support. Effectiveness 
trials, in contrast, assess effectiveness under more typical school conditions. Note that 
the study of Reading Recovery (Schwartz, 2005) is technically an effectiveness trial; 
however, all the Reading Recovery interventionists received extensive training, which 
is typical of an efficacy study. 

2. A pseudoword (or nonsense word) is not a real word but rather one created to test a 
student’s ability to decode unfamiliar words. It consists of a string of letters that resent- 
bles a real word (for example, fot, vap, muke). Thus, students’ knowledge of sight words 
will not influence scores on pseudowords measures. 

3. See note 2. 

4. The four areas of reading performance are those identified in U.S. Department of 
Health and Human Services (2000). 

5. Potentially positive findings, or findings that approach but do not quite reach the tra- 
ditional level of significance, should be interpreted with caution because they may 
have been a result of chance. 

6. The study of Reading Recovery (Schwartz, 2005) is technically an effectiveness trial, 
but all the Reading Recovery interventionists received extensive training, which is 
typical of an efficacy study. 

7. Academic vocabulary represents a set of words that are used in academic classrooms 
and text much more often than in everyday social and informal settings. Academic 
vocabulary words include both general academic words and domain-specific words. 

8. None of the eligible studies included an overall reading achievement measure. 
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